Active Sensing for Opponent Modeling in Poker

One approach to designing an intelligent agent capable of winning competitive games such as Texas hold’em poker is to use opponent modeling to learn about an opponent’s behavior, then exploit that knowledge to maximize long term winnings. However, opponent modeling can suffer from several problems, including slow convergence due to a lack of a priori knowledge, noisy or dynamic opponent behavior, and possible reverse exploitation through the “get taught and exploited” problem. One potential solution to these problems is active sensing, whereby an agent actively decides how and when to gather information to revise its knowledge, rather than simply relying on whatever observations are produced during its routine actions. In this paper, we describe an active sensing agent for Texas hold’em poker introduced into the 2011 ACPC, which finished the best of known opponent modeling agents in the heads up limit competition. In an empirical evaluation, we demonstrate the benefits of active sensing, including improved winnings and less uncertainty. We also describe how active sensing can be extended to other types of opponent modeling agents, and identify key areas of improvement for our agent.